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'SI March 198 0 


MEMORANDUM FOR: Deputy Director for Administration 

STATINTLfrom: 


STATINTL 


SUBJECT : 



STATINTL 


m\m 


pi 

t.K 
Staff 

Tf 


tewing 


r o w a 

er his consul ta^Wn^nen^^fnc^wu^Tiai led 
hief of the Psychological Services 
offered to write this letter after 
t Report and the IG Consultants’ 

The letter was not solicited in any way. 


STATINTL 


2 , I have reviewed this letter carefully and consider 
that it speaks for itself. I view it as an important adjunct 
to the more extensive rebuttal to the IG Draft Report on PATB. 
I also request that it be forwarded to the Inspector General 
and to the DCI and DDCI as a part of QMS response to the IG 
Report and IG Consultants’ Report on PATB. 


STATINTL 


Attachment 
(4 copies) 
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STATINTL 


STATINTL 


March 24, 1980 


STATINTL 


This letter is a follow-up to my visit wh 
me to review the I.G. reports prepared by 

STATINTL two 

I have organized this letter into four parts: 

I. Their criticisms of the Strong-Campbell Interest 
Inventory. 


I think they are off the mark here. They 
seem not to have stayed current with new 
developments . 

II. Their evaluation of the Professional Applicant 
Test Battery (PATB) , and its use. 

Their concerns are more those of isolated 
academicians, who can be critical and then 
go home, rather than of involved professionals 
who have to make a selection system work in 
the world of reality. 

ill. Some specific comments on the overall report. 

I think they were superficial and sloppy, 
also hostile. 


STATINTL 
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IV. A suggestion for future improvement in the Agency's 
selection policies. 

I have suggested what I think might be done 
next . 

I hope this letter and report is helpful. STATINTL 


DC/ac 
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I. Their criticism of the Strong-Campbell Interest 
Inventory (SCII) . 


STATIN TL 


STATINTL 


my reaction 

face^^n^cnarg^o^neiri^defensj^^^^^dor^t know anyway 
around that other than being straightforward about it, 
citing some contrary evidence, and invoking the judgments 
of others that have appeared from published reviews. 

critical of the SCII in several 
placS^Tu^TnH^Trepo^^^^Tor example: 

"No validity studies have been done on the SCII. It 
appears as though the psychologists [PSS staff] have 
assumed that the validity data accumulated for the 
old Strong Vocational Interest Blank , which is, by 
the way, completely inadequate and unconvincing, can 
be applied to the SCII. It cannot be: the SCII is 
a new instrument with quite different characteristics 
from the old Strong Vocational Interest Blank ." 

Appendix 1, p. 38-39 

This is an example of the gratuitous remarks that so 
often weaken their report. First, to accuse a professional 
staff of being confused about a major revision of a test 
in their battery is a serious charge of incompetence, one 
that could quite easily have been shown not to be true by 
asking a few simple questions of the staff. Why didn't they 
ask them? 

Second, to dismiss out of hand the accumulated mass of 
evidence on the Strong Vocational Interest Blank by the throw- 
away comment ... "which is, by the way, completely inadequate 
and unconvincing" is a real zinger. 

That the investigators themselves may not be of a single 
mind here is indicated by a second quote on the SCII, appearing 
later: 


"The SCII is a standardized instrument which provides 
validity data in its manual to establish what it is 
appraising . " 

Appendix 2, p. 9 
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This kind of inconsistency is usually found in reports 
produced by two investigators who do not consult with each 
other, or read what the other has written. It is sloppy 
and unprofessional. 

The SVIB is the most thoroughly documented non-cognitive 
test in existence. Two massive Handbooks are available 
(Strong, 1943; Campbell, 1971) , an 18 year followup study 
(Strong, 1955) , books by others on the inventory (Darley, 

1941, Darley and Hagenah, 1955), a string of Technical Manuals 
(Strong 1959, Campbell 1966, 1968, 1974, 1977) and 1,521 
technical publications cited in the Buros Mental Measurement 
Yearbook Series. 

Of course, not all of the publications are positive and 
supportive. Some are critical, and one of the characteristics 
of the SVIB-SCII system is that we have constantly tried to 
improve it, using as part of our guidance the useful criticism 
of our professional colleagues. Since I have been in charge, 
the inventory has been revised three times: in 1966, 1969, 
1974, with another revision coming in 1981. Consequently, 
as Thorndike and Hagen point out, the current version of the 
inventory is not the same as the older version; in the new 
editions, we tried hard, successfully I think, to retain the 
beneficial aspects of the earlier forms while incorporating 
improvements suggested by ongoing research. Consequently, 
the SCII is superior to its predecessors in at least the 
following ways: 

1. It is shorter, by virtue of eliminating invalid 
or outdated items, and those items possibly 
offensive to various groups. Its reading level 
has been lowered. The instructions and items 
have been polished. 

2. It combines the earlier separate forms for men 
and women into a single booklet that can be used 
by both sexes. 

3. It has a theoretical orientation, allowing for 
generalizability of results; the earlier versions 
were completely mechanistic, atheoretical instruments 
that forced the user into being his/her own 
theoretician. 
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4. It has many more scales for both men and women; 
these scales have virtually the same psychometric 
qualities as those on the earlier form. Many are 
better. 

5. It now has two basic kinds of scales, reflecting 
the two most common scale construction techniques-- 
internal consistency and criterion-related~-so 
that the user can take advantage of the benefits 

of each. 

6 . The graphic representation of the scores is more 
sophisticated, and a long computer-produced 
narrative report is now available. 

7. The versatility of the computer has been used to 
markedly reduce scoring inaccuracies. 

8. The interpretive information provided to the user 
has been increased at least tenfold. The Manual 
has increased in size by a factor of three or four. 
Far more information, both interpretive and 
cautionary, is available than ever before. 

The accumulated mass of evidence on this inventory 
shows clearly that: 

1. People who enter different occupations have dif- 
ferent interests. 

2. These interests can be tapped by quantifiable 
methods, and then described on dimensions ranging 
from "high to low. " 

3. Individuals with "high" interests in some vocational 
area are more likely to enter that area, and remain 
there happily employed than are those with lower 
interests. 

4. When good measures of specific occupational per- 
formance are available, and they hardly ever are, 
some interest patterns show positive correlations 
with performance; others do not. 
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5. People with "low" measured interests in some 
occupational area are less likely to enter that 
area. If they do enter, they are more likely 

to leave, or express occupational dissatisfaction, 
or perform the occupation in some unusual manner 
than are people who score "high." 

6. Interest test scores are stable, especially for 
adults and especially over short time spans (of 
2-3 years) . Even among teenagers tested over 
20-40 years lately, the average degree of stability 
is startling. 

I could list another 20-30 findings, but these general 
ones serve the purpose. To dismiss this accumulation of 
50 years of research as, ..."by the way, completely 
inadequate and unconvincing. . . " is the mark of an investigator 
who is concerned with some issue other than a thoughtful, 
professional review of the Agency's test battery. 

Should any reader wish to pursue this, the SVIB-SCII 
has been thoroughly and repeatedly reviewed in the prestigious 
Mental Measurements Yearbook Series, with the latest reviews 
appearing in 1979. Here are some direct quotes from those 
reviews, as well as some from other sources. 


"The latest (1959) manual for the SVIB is well organized 
and provides detailed information on the rationale, 
construction, reliability, and validity of the SVIB, 
as well as an informative section on use and interpreta- 
tion of the test in counseling and personnel work. 

Recent major research studies ... which are summarized 
only briefly in the Manual, now make it clear that 
the SVIB is useful for predicting membership in given 
occupations over long periods of time, and of moderate 
value for predicting success within a few selected 
occupations ... In comparison with competing tests, the 
major liability of the SVIB would seem to be the 
complex problem of interpreting a profile based 
on such a large number of scales, although the extensive 
work which has recently been done with group keys 
has alleviated much of this difficulty. In any case, 
there is still little doubt that the SVIB remains as 
the best constructed and most thoroughly validated 
instrument of its kind." p. 1304 

Alexander W. Astin 
Director of Research 
American Council on Education, 
The Sixth Mental Measurements 
Yearbook, 1965, 
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"During the 10 year period since the previous review, 
the vitality of this inventory has continued undiminished. 
Important publications include Strong's Vocational Interests 
18 Years After College (1955), Darley and Hagenah ' s 
Vocational Interest Measurement (1955) and a new manual. 

To these should be added such further interpretive aids 
as Layton's monograph (1955), substantial treatments in 
testing books, such as that by Super and Crites, and 
upwards of a hundred research studies in periodicals and 
dissertations .. .All in all, the Strong remains a solidly 
based but rather complex inventory suitable mainly for 
older adolescents and adults considering higher level 
occupations..." p. 1304-5 


Edward J. Furst 
Prof, of Psychology 
Ohio State University, in 
The Sixth Mental Measurements 
Yearbook, 1965 


"Strong published his inventory in 1928, offering as 
evidence of validity the fact that the scores distinguished 
middle-aged men of different occupations from each other... 
After publication of the test, Strong continued to accumulate 
evidence by following men for 20 years or more and by 1954 
was able to verify that the test indeed makes fairly valid 
predictions over a long period." p. 137 


Lee J. Cronbach 

Prof, of Educational Psychology 
Stanford University, in 
Essentials of Psychological 
Testing, 3rd Ed. 

Harper & Roe, 1970 
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"The (1969) revision has been handled with the 
circumspection that befits a highly respected institution. 
In a time when all established institutions are buffeted 
by winds of change, Campbell has apparently sought a 
middle course between orthodoxy and radicalism. He 
seems to be trying to preserve the structure of the 
instrument that Strong built, so that the hierophants 
can continue their comfortable reliance on accumulated 
data, personal experience and familiar ritual. At the 
same time, he has modernized and streamlined the instrument 
has made it more adaptable, and has laid a foundation for 
more extensive innovations in the f uture ... Until the 
scheduled Handbook for the SVIB [now published, 1971... 
David Campbell] or future handbooks enlighten counselors 
on the advantages of the SVIB over expressed interests, 
this reviewer is reluctant to recommend the use of the 
SVIB--or any other interest inventory." p. 1464 

Martin R. Katz 
Senior Research Psychologist 
Educational Testing Service, in 
The Seventh Mental Measurement 
Yearbook, 1972 


"This is a paper-and-pencil inventory of interests 
applicable to late adolescents and young adults .. .With 
the mass of information available, users and interpreters 
of the test should have some training in psychological 
testing and spend some time with the manuals and the 
Handbook. The Handbook appears to be the most complete 
document of its kind ever assembled ... Both the original 
author, E.K. Strong, and the current authors, especially 
D. P. Campbell, have shown themselves willing to tackle 
the complex and frustrating task of providing as complete 
as possible information for their test. They have 
succeeded better than most. The authors are aware of 
the criticisms of their efforts and are to be complimented 
on their efforts to correct deficiencies where they agree 
and to make the rest of us understand where they do not 
agree . " 


Charles J. Krauskopf 
Professor of Psychology 
University of Missouri, in 
The Seventh Mental Measurement 
Yearbook, 1972 
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"Over a period of 40 years, the SVIB has been revised 
and refined extensively, and it is among the most useful 
and successful structured inventories available today... 

The most recent revision of the men's form of the SVIB 
contains carefully validated and up-to-date scales for 
54 specific occupational groups. The Handbook for the 
Strong Vocational Interest Blank summarizes results from 
over 100,000 adults in over 400 occupations, some of 
which were tested four times over 40 years. Given the 
success of the SVIB in the realm of vocational interest 
measurement, it is understandable that the empirical 
method of contrasted groups has also been attempted in 
the realm of personality measurement as well." 

p. 388-389 

Jerry Wiggins 
Professor of Psychology 
University of Illinois, in 
Personality and Prediction: Principle 
of Personality Assessment 
Addison-Wesley , 1973 


"Although all of the interest inventories that use 
normative or occupational scales--the several versions of 
the Strong Vocational Interest Blank (SVIB) , the Minnesota 
Vocational Interest Inventory (MVII) , and the KOIS--have 
extensive data in their manuals on their ability to distin- 
guish between persons in different occupations (a form of 
concurrent validity) , really only the SVIB has had 
extensive investigation of its predictive validity. The 
best known is Strong's 18-year follow-up (Strong, 1955), 
but a number of other studies have been performed and are 
reported in convenient form by Campbell (1971) . Strong con 
eludes that persons are three times as likely to be in an 
occupation on which they made a T score of 45 or higher 
than in an occupation on which they scored lower than 25. 
Campbell concurs in his review of other follow-up studies, 
although he suggests that when the inventory is taken by 
high school boys, it is used more effectually to suggest 
general direction of career rather than specific occupation 
Berdie (1960) and Super and Crites (1962) review much of 
the same material and draw similar conclusions." 

Donald Zytowski 

Journal of Counseling Psychology, 1976 

Vol. 23, No. 3, p. 221 
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"Information about these inventories is now voluminous 
and scattered throughout the literature. Campbell's (1971) 
Handbook for the Strong Vocational Interest Blank (SVIB) is 
the largest and most comprehensive summary of the evidence 
about any single inventory. This companion volume to 
Strong's (1943) Vocational Interest of Men and Women pro- 
vides an extensive account of the history of the develop- 
ment of the SVIB, including the most recent developments — 
the development of the basic scales for men and women, 
administrative indices for detecting errors and response 
bias, new occupational scales for specific occupations, 
and new and old data about the retest reliability and validity 
of both the old and new scales. In addition, the appendices 
provide 180 pages of empirical detail. Even a casual reader 
would conclude that the SVIB must be the most well validated 
inventory in existence." 


John L. Holland 

Handbook of Industrial and 

Organizational Psychology, 1976 
Chapter 12, p. 523 


"Recognized as the paragon of applied behavioral 
measures and widely acclaimed as the bellwether of career 
counseling and personnel selection, the several editions of 
the Strong Vocational Interest Blank have a venerable his- 
tory and reputation, dating back to the late 1920s and 
spanning the subsequent decades to its most recent revision 
in 1974. Literally thousands of studies have been conducted 
on the SVIB, and millions of copies have been sold. It has 
launched and sustained careers, as well as assessing them, 
and it has served as a source of master's theses and doctoral 
dissertations. It seems almost presumptuous, therefore, to 
review the Strong, but such is the conscience of the testing 
community. Even the Strong must bear the scrutiny of 
critical analysis, although it may be difficult to find 
other than picayune flaws in it. Particularly is this true 
of the current edition, the Strong-Campbell Interest Inven - 
tory , which represents the culmination of almost 50 years 
of unparalleled ' dustbowl empiricism' on the measurement of 
vocational interests." 


John O. Crites 
Professor of Psychology 
University of Maryland, in 
The Eighth Mental Measurements 
Yearbook, 1976 
Volume II, p. 1621 
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"The new SCII is well constructed. The reliability 
and validity figures reported for the new SCII scales are 
comparable to those reported for the SVIB scales." 


"Campbell and his co-workers not only have merged 
effectively the male and female forms, but also have 
improved substantially the interest inventory by eliminating 
undesirable items from the item pool, by adding a large 
number of new Occupational Scales, and by providing a 
theoretical structure for the empirical scales." 

Richard W. Johnson 
Measurement & Evaluation in 
Guidance Journal, 1976 
9 (I) : 4 0-5 


"The SCII appears to the reviewer to be the best 
vocational interest inventory available." 

Robert H. Dolliver 
Professor of Psychology and 
Counseling Psychologist 
University of Missouri, in 
The Eighth Mental Measurements 
Yearbook, 1978 
Volume II, p. 1626 


"The convergent and divergent construct validity of 
the Strong-Campbell Interest Inventory and the Vocational 
Interest Inventory ,.. .was examined separately in samples 
of male and female college counselees. The validity of 
the SCII General Themes was strongly supported while 
minor weaknesses in certain Basic Interest and Occupational 
scales were revealed." 

(abstract) 

"Construct Validity of the Strong- 
Campbell Interest Inventory. . . " 

Journal of Vocational Behavior, 10, 1977 
pp. 187-195 

Patricia W. Lunneborg 
Professor of Psychology 
University of Washington 
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From their criticism, 

out-of-step with the rest o^^^i^pr^essl^iu 


seem to be 


STATINTL 


II. The investigator's description of the use of the 
Professional Applicant Test Battery (PATB) . 


Before I read the I.G. report, I spent several hours 
looking through applicant files, reading the staff summaries. 
They seemed appropriately done; if anything, restrained. 

The summaries were cautious, staying close to the data. 


STATINTL 


I thought to myself, "This is the kind of timid be- 
havior created by bureaucracies; because they may be called 
on to defend these summaries, the psychologists are offering 
little more than an explicit description of the test scores — 
no additional hunches, educated guesses, or clinical wisdom 
that could help the employing manager go beyond these scores. 


Consequently, I was surprised to read 
H9 characterizations of .the PATB suramarle 



STATINTL 


"The [patb] narrative report is partly description, 
partly prediction, and partly fantasy." 


Appendix 1, p.4 

"...we have been extremely troubled by the unrestrained 
enthusiasm with which the psychological staff has 
promoted the operational use of the PATB test scores 
for selection and placement of personnel in the agency. 

Appendix 1, p.37 

"The failure of PSS to recommend an applicant is 
equivalent to a "kiss-of-death" for that applicant 
in some of the units. This makes the recommendation 
section of the narrative report even more troublesome 
because PSS makes its recommendations with a level 
of confidence and finality that is not supported by 
the validity and reliability of' the data." 

Appendix 2, p.7 
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These accusations of "partly fantasy," "unrestrained 
enthusiasm," and "kiss-of-death" are difficult charges 
for the psychological staff to defend themselves against; 
if they offer up contrary examples of their work, the cases 
can easily be dismissed as biased selection. If they simply 
say, "No, that's not true," they come across as defensive. 

There is really no way they can answer, unless the reader 

of the report has the opportunity to look through some of STATINTL 

the applicant folders. 

that opportunity, and after readinq^ BpS ^^fl 
report, which seemed so at odds wi^^m^ow^™ 
pe^e^S^n, I went back and simply grabbed a handful of 
folders and have quoted the summary statement from several 
of them below. While these don't constitute a perfectly 
random sample, the choice was haphazard. Clearly, these 
reports constitute something other than "unrestrained 
enthusiasm," or "fantasy." (Some of these summaries are 
from case folders of agency employees who came in to be 
tested for career development purposes; others are from 
applicant files.) 

Case #1 

"Candidate's non-verbal abilities are suitable for 
Agency employment, but his verbal skills are 
evidently weak, by Agency standards. His work 
attitudes are positive, and he claims some relevant 
career experience. OL may have an interest." 

Case #2 

"Candidate's scores range from poor to above 
average. OIA may have an interest." 

Case #3 

"Candidate's test abilities range from poor to 
average, and are, overall, weak, when compared 
with those of most Agency personnel. However, 
in view of his education, and obvious interest 
in mechanical activities, NPIC may want to con- 
sider him for a technical position." 
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Case #4 


"Candidate appears to be a bright young lady 
(average by Agency professional standards) 
who has some potential for development along 
professional or at least semi-professional 
lines. She has indicated that she would 
eventually like to make use of her undergraduate 
training in biology and chemistry in some way, 
does well with mathematical problems, and enjoys 
physical activity. In addition to perhaps 
receiving training as an investigator, are there 
any technical or scientific assignments in which 
she might be given an opportunity to gain some 
knowledge and make a contribution?" 

Case #5 

"On the basis of these few test scores, the 
candidate would appear to be rather less qualified 
for Agency employment than other applicants, par- 
ticularly in the realm of verbal abilities." 

Case #6 


"Candidate's most obvious weaknesses are in the 
verbal and writing skills areas, which he may 
wish to address through some Agency in-house 
training aimed at strengthening such skills. 

His measured interests bear little relationship 
to Agency positions and are therefore of limited 
value in determining beneficial career assignments. 
A determination of his ability to handle a pro- 
fessional assignment would depend upon the assign- 
ment he would have, the demands made upon him, 
his level of interest, and past work assessment. 
Compared with other Agency employees with his 
level of education, candidate's abilities are 
generally average to slightly below average." 
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Case #7 

"This is a highly confident, calm, collected, and 
self-assured type person who perceives himself 
to be very much the leader in any group. In 
terms of work he is very willing to function 
under conditions that require stringent, security 
measures and involve a constant change of pace 
and schedule. . . 

It is not clear what may be contemplated [in terms 
of OS, for this applicant]. If it is the position 
of investigator (and [applicant] agrees) , the work 
attitude/temperament indices as well as the high 
score areas of the PATB would tend to support con- 
sideration; i.e., he has certainly the intellectual 
ability needed and the other indices suggest good 
stress tolerance and interpersonal skills." 

Case #8 

"Candidate's measured interests have a technical- 
realistic direction: candidate shares the 
interests of foresters, veterinarians, police 
and highway patrol officers, registered nurses, 
medical technicians, and pharmacists. 

Candidate appears to have potential for development 
along professional lines in the OS ... Candidate 
indicates wish to pursue a graduate level academic 
program, but hopes to continue working while he 
does so." 

From test feedback interview: 

"Candidate was especially interested in his interest 
patterns, which reflected quasi-medical interests 
but supported earlier decision to drop his pre-med 
studies to take up anthropology... 

I encouraged him to seek more information in the 
Agency in [a special interest area] and suggested 
[a career planning] book." 
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Case #9 


"Candidate's tested abilities range from poor 
to excellent, and most fall into [the Agency's] 
average range. His work attitudes and work 
interest patterns reflect his occupational 
choices consistently. He seems well-suited for 
a job in which he can use his technical interests 
and skills. OWI may have an interest." 

Those summarie^h^^^^me^b^th^severe criticism 
leveled at them My own judgement 

is that the summar^l^^^^®^SI^^Hff!^^^ll-done, given the 
number of applicants, and limitations of the staff. The 
routine nature of the reports strongly suggest that much 
of this work could be computerized. 


III. Some further, specific reactions. 


Job Analysis 

The approach they recommend — that of atomistically 
analyzing each job, then developing and validating a test 
(or an equation) against each job element for a range of 
various demographic categories — white, black, Hispanic, 
male, female — and then using the tests as selection 
devices is simply too overwhelming and burdensome for 
most organizations to use, especially if they have large 
numbers of jobs with only a few people each in them, or 
if they have jobs that continually change in character. 

The permutations involved in validating tests for 5 
or 6 demographic categories for even as few as 8 or 10 
jobs quickly become overwhelming, even if the various 
test population are available, which they are not. 

Textbook academicians who have been recommending 
these systems for years by now should know better; the 
systems simply haven't worked. The incisive proof of 
that is the pudding that the academicians themselves 
have not eaten; no university that I know of uses these 
methods to select their own students, staff, or faculty. 


STATINTL 
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Reliability 

Most of their long discussion on the topic of 
reliability is elementary, more suitable for an entry 
level course in statistics than for a real world report. 

It is a red herring thrown into a report that seems to 
have trouble focusing on the real issues. 

• In the first place, the tests in use have 
suitable reliabilities. 

• In the second place, even if they were not 
reliable, the selection ratio is so high (those 
who apply vs those who are hired) that some error 
in the measures would not hurt much anyway. With 
this selection ratio, the cutoff point can be far 
above any wobble created by unreliability. 

• In the third place, because of what we know about 
scale length, item intercorrelations, and 
reliability, virtually any sensible psychologist 
can now create a reliable scale. Six to eight 
Likert-type items, with a reasonable homogeneous 
core of content, will produce a reliable scale. 
Once you give up inkblots, or TAT's, or Draw-A- 
Person tests, reliability is hardly ever an issue. 
It certainly isn't here. 

"Got-chas" 


The tenor of the I.G. report is quite critical, and 
one of the themes running through the report is the authors' 
proclivity to use catchy, belittling lines of criticism 
that are difficult to defend against: 

"The narrative report is partly description, partly 
prediction, and partly fantasy." 

"kiss-of-death" 

"unrestrained enthusiasm" 

" [The Policy] has disappeared into the mists of 
antiquity." 
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These little "got-chas" make good reading for the 
outsider and create in the authors a momentary sense 
of headiness for their devilish cleverness, but they 
create an unfair dilemma for the professional who is 
thus accused. How does one defend oneself against an 
unspecified charge of "partly fantasy?" Without knowing 
on which cases the charges were based, the professional 
can only answer "it is not fantasy.'" which is hardly 
convincing. 

Further, the tone of the report is basically nasty 
and hostile; for example, it automatically assumes that 
the PATB is anti-female and anti-minority. 

Experienced outside consultants would be aware that 
the professional staff is going to have to answer each 
criticism, and would couch the criticism in specific, 
non-emotional terms, perhaps even with a suggestion for 
improvement, instead of flinging these generalized grenades 

Level of Sophistication 


In general, the report is written below the level of 
sophistication of the professional staff. The discussions 
of job analysis and reliability , above, are instances of 
that. 


Another instance is their comment: "The selectivity 
may well have worked to the advantage of the Agency, but 
it tends to be disastrous for validation research." 
(Appendix 1, p.12.) To suggest that the ends of the 
researchers are more important than the overall operational 
needs of the Agency is a naive comment. 

Assessment Center Methodology 

The investigators give no nod at all to the important 
developments in the assessment center mode of assessment, 
which attempts to assess "the whole person" rather than 
tie the selection decision to one or two specific test 
scores. In the current situation, where the Agency is 
trying to select people for careers rather than for 
specific jobs, this global approach makes more sense. 
Instead, the I.G. report, with its emphasis on job analysis 
molecular job components, and selection formulas argues for 
regression not progression, on this point. 
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STATINTL 


Summary 


The flavor of this report comes across as having 
been written by someone: 

(1) who was active in 1940-1950's testing psychology , 
but who has not kept up on new developments , 
such as assessment centers and test revisions, 

(2 ) who has had little practical experience in 
actually installing and operating selection 
systems in current organizations — the report 
essentially recommends a 1940/50 academic 
solution to the selection dilemma, a solution 
which, despite 30-40 years of recommendations 
in the textbooks, has never really worked any- 
where that I am aware of, 

( 3 ) who have mainly had a graduate student audience 
who were unempowered to answer back to either 
unfairness or nonsense — thus the report is 
peppered with snide, little digs that the authors 
would have trouble defending to knowledgeable 
professionals who disagree with them, 

(4) who has an EEOC axe to grind — that axe may well 
be worth grinding, but that cause is not well- 
served by the quality of this report. 


IV. A suggestion for an overall Selection Policy . 


My main criticism of the Agency's selection procedures STATINTL 
is the apparent absence of any overall guiding policy for 
the select ion of new professionals, a point that 
HHHialso made. The Agency's "Selection 
sucx^£u^!u; is now, flows from a combination of the myriad 
decisions made: 

• by managers asking for certain types of 
individuals, 

• by recruiters in their efforts, 
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• by investigators doing background checks, 

• by psychologists in their testing roles, 

• by various administrative constraints , such as 
affirmative action concerns, and probably 
adversely, 

• by the system that frequently forces a lengthy 
delay between time of initial interview and final 
job offer. 

The net result is that the selection policy makes 
itself through the interaction of these forces. 

(A caveat here: the Agency, by its nature, is not 
an open institution; consequently an outsider gropes 
blindly for solid data about what is happening. The 
preceding paragraph is based mostly on small wispy clues, 
hunches, and educated guesses, not on hard data. The 
reader should be cautious; I may be wrong.) 

An explicit, comprehensive policy for the selection 
of the Agency's new professionals would be a great help 
in deciding precisely what selection techniques should be 
used. 


The policy should come from on high; the implementation 
should be the responsibility of the Psychological Services 
Staff. 

My suggestion would be that at least the following 
points be addressed in an overall policy. 

I. TALENT : The Agency wishes to hire the best talent (s) 
available for its mission. Because a range of 
talents is necessary, multiple, overlapping categories 
should be considered. Here are the most likely 
categories: (again, ' from the viewpoint of a 

relatively naive outsider) 

A. Analytical 

1. A general facility with words and 
numbers . 

2. The ability to see themes in murky 
material . 


Approved For Release 2002/01/25 : CIA-RDP00-01458R000100130014-3 



- 19 - 

Approved For Release 2002/01/25 : CIA-RDP00-01458R000100130014-3 


3. The ability to conduct numerical 
analyses . 

4. The ability to analyze and summarize 
written material. 

5. The ability to forecast future trends 
from current material. 

B. Technical Training 

1. A solid background in some area relevant 
to the Agency's mission; perhaps 

(a) mathematics/computer sciences 

(b) physical sciences 

(c) economics 

(d) political science 

(e) agriculture 

C. Communications Skills 

1. The ability to write clearly and crisply 

2. The ability to report orally, both one- 
on-one, and as a briefing officer. 

3. Persuasive ability as necessary. 

D. Language Skills . 


II. STABILITY : The superficial image of the Agency is 
one of a James Bond world of intrigue, danger, 
excitement, sex and money. Consequently, the 
applicant pool has a disportionate number of 
shallow, unstable, occasionally even psychotic, 
thrill-seekers. The selection procedures should 
weed them out. But balanced against this is the 
hard fact that the Agency needs to attract people 
who are willing to live unusual, atypical lives; 
for them part of the motivation is surely the 
"sense of differentness" involved. Consequently, 
policy here has to walk the fine line between 
the following two orientations (they are not 
necessarily either/or) : 
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1. Personal soundness : 

(a) Strength of convictions in the 
accepted mores of our culture 
(e.g., that elected officials are 
ultimately responsible) . 

(b) Tolerance for stress and ambiguity. 

(c) A generally optimistic view of life. 

(d) The ability to grow, not to stagnate. 

(e) An absence of mental, legal, finan- 
cial, or physical problems. 

2. A certain venturesomeness 


(a) Willingness to seek and experience 
change . 

(b) Calmness in the face of hazards. 

(c) Flexibility in the area of personal 
comfort. 

(d) A generally positive orientation 
toward risk. 


III. MANAGERIAL POTENTIAL : Presumably the Agency is 
hiring for careers, not entry level positions, 
and because managerial talent comes mainly from 
within, the leaders of the future are now being 
hired. Consequently, some thought should be 
given to the future in the selection process. 

At least the following factors may be involved. 

1. A willingness to seek more reponsibility . 

Those who move up the administrative 
ladder are more willing, even eager, 
to accept additional responsibility 
and suffer the related consequences. 
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2. The ability to follow and learn from 
older models. 

Following a "mentor" is one of the 
most common routes to top management. 
Those capable of selecting and being 
selected by people who are excellent 
teachers will more often be found 
in the higher reaches of management. 
(Psychologists currently know virtually 
nothing about how to select for this 
characteristic. ) 

3. An air of "competitiveness." 

Good leaders are usually competitive, 
physically active, willing to risk 
losing for the sake of winning. 


IV. ADMINISTRATIVE CONSTRAINTS : There are certain 

external constraints that affect hiring practices; 
they, of course, must be recognized. At the 
moment, for example, affirmative action policies 
must be part of the overall selection policies. 

V. " FOREIGN" ORIENTATION : I don't really know what 
I am talking about here, but the point seems too 
important to ignore. Because of the nature of 
its charter, the Agency needs to be an inter- 
nationally oriented institution. Consequently, 
other characteristics being equal, those applicants 
are most desirable who have: 

1. Lived, worked, or traveled extensively 
overseas, or, 

2. Have spend years studying other cultures, 
or 

3. Have been raised by an internationally 
oriented family, or by a family from 
another culture, or, most preferably, 

4. Some combination of the above 3. 
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SUMMARY 

I am aware of the arrogance exhibited here in pro- 
posing an Agency selection policy on the basis of a one- 
day visit. Still, the absence of such a policy seems 
to be at the heart of the current dilemma. With such a 
policy, the necessary implementation could be done, and, 
in theory at least, some evaluation research could be 
conducted. As it stands now, those responsible for doing 
the work are at the mercy of outside experts who, free 
from the constraints of suggesting something that might 
work within the Agency's desired policy, are free to take 
potshots at the current procedures without worrying about 
practical reality. 
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